Update variational inference interface to match `AdvancedVI@0.6` #2699

Red-Portal · 2025-10-22T08:33:49Z

This PR does the following:

The interface to AdvancedVI is updated to match the new v0.6 version.
The way vi calls optimize is slightly changed so that algorithms that expect an unconstrained space can be used without being aware of Bijectors. In particular, the user is now expected to provide a q <: Bijectors.TransformedDistribution. vi will then unwrap the underlying unconstrained distribution, which is passed to AdvancedVI. Once the algorithm terminates, the result will be wrapped back and then returned. (See also this PR for a more detailed explanation.)
The main file is renamed from VariationalInference.jl to Variational.jl to match the submodule name Variational.
The deprecated v0.2-era interface is removed.

github-actions · 2025-10-22T08:40:01Z

Turing.jl documentation for PR #2699 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2699/

codecov · 2025-10-22T08:44:32Z

Codecov Report

❌ Patch coverage is 0% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.51%. Comparing base (ea9bb54) to head (d83148b).

Files with missing lines	Patch %	Lines
src/variational/Variational.jl	0.00%	27 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (ea9bb54) and HEAD (d83148b). Click for more details.

HEAD has 7 uploads less than BASE

Flag BASE (ea9bb54) HEAD (d83148b)

24 17

Additional details and impacted files

@@              Coverage Diff              @@
##           breaking    #2699       +/-   ##
=============================================
- Coverage     86.54%   57.51%   -29.03%     
=============================================
  Files            21       20        -1     
  Lines          1412     1410        -2     
=============================================
- Hits           1222      811      -411     
- Misses          190      599      +409

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mhauru

Thanks @Red-Portal.

Since this changes the user-interface, could you please set the target branch of this PR to be breaking? We use that branch to collect breaking changes into a single, larger release. Currently breaking points at the same commit as main, so you shouldn't have to mess with git history for this.

In that same process, could you also please add an entry in HISTORY.md describing to users all interface changes (arguments, return values, behaviour) and what they can do to adapt to them. Maybe a code example of "this is how you used to call vi, this is how you do the same thing now", that sort of thing.

src/variational/Variational.jl

src/variational/VariationalInference.jl

src/Turing.jl

test/variational/advi.jl

test/variational/vi.jl

…advancedvi_0.5

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

HISTORY.md

penelopeysm · 2025-10-24T10:32:22Z

I'm actually sorry to butt in on this, but are there new features available in 0.5, or is it just an interface change?

Red-Portal · 2025-10-24T14:42:30Z

@penelopeysm No new features.

penelopeysm · 2025-10-24T15:54:45Z

Okay. The reason why I asked is because this is an interface change for Turing and that is kind of annoying for end users if they don't get any benefit. I wonder in that case if it is worth doing either:

Not bumping the compat in Turing, waiting for the next release of AdvancedVI that has new features; or
Mapping the existing Turing interface to the new AdvancedVI interface, so that it's not a breaking change. I don't know how possible this is.

Other than that, I second everything that @mhauru brought up here, so I won't also review the details of this.

Co-authored-by: Markus Hauru <markus@mhauru.org>

…ncedvi_0.5

… into bump_advancedvi_0.5

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Red-Portal · 2025-11-21T18:15:33Z

Hi @penelopeysm @mhauru @yebai , I've updated the code to match the new v0.6 version. Let me know if you have any comments!

mhauru

Hi @Red-Portal, thanks for this. The code looks good, just some comments here and there on documentation, since I think it's important to get it right for such a big change.

mhauru · 2025-11-27T09:55:31Z

src/variational/Variational.jl

+If the chosen variational inference algorithm operates in an unconstrained space, then the provided initial variational approximation `q` must be a `Bijectors.TransformedDistribution` of an unconstrained distribution.
+For example, the initialization supplied by  `q_meanfield_gaussian`,`q_fullrank_gaussian`, `q_locationscale`.


Just trying to understand the meaning of this: What would be choices of q that would not be valid for an unconstrained-space-requiring algorithm?

Something like LogNormal, Dirichlet, and of course all discrete support distributions.

src/variational/Variational.jl

mhauru · 2025-11-27T10:09:42Z

test/variational/advi.jl

 using Turing
 using Turing.Variational

-@testset "ADVI" begin


I often find it helpful that all the tests in a file are wrapped in a single big testset. Makes the output easier to read. Did you have a particular reason to prefer otherwise here?

It's already wrapped in runtests.jl by variational_algorithms and we're testing essentially all of the functionalities in the Variational module. So I think an additional @testset wrap is redundant at the moment.

HISTORY.md

mhauru · 2025-11-27T10:17:09Z

HISTORY.md

+
+  - The default hyperparameters of `DoG`and `DoWG` have been altered.
+  - The deprecated `AdvancedVI@0.2`-era interface is now removed.
+  - `estimate_objective` now returns the value to be minimized by the optimization algorithm. For example, for ELBO maximization algorithms, `estimate_objective` will return the *negative ELBO*. This is breaking change from the previous behavior where the ELBO was returns.


I think estimate_objective isn't exported by Turing. Is there some other way in which this change is visible to Turing users?

Not in an export sense, but the tutorial here brought it up, so I thought it would be nice to state explicitly.

HISTORY.md

Co-authored-by: Markus Hauru <markus@mhauru.org>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Co-authored-by: Markus Hauru <markus@mhauru.org>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

update vi interface to match AdvancedVI@0.5

ea69430

Red-Portal requested review from mhauru and penelopeysm and removed request for penelopeysm October 22, 2025 08:33

github-actions bot assigned Red-Portal Oct 22, 2025

revert unintended commit of runtests.jl

86ee6dd

Red-Portal requested review from penelopeysm and yebai October 22, 2025 08:35

mhauru requested changes Oct 23, 2025

View reviewed changes

src/variational/Variational.jl Show resolved Hide resolved

src/variational/VariationalInference.jl Outdated Show resolved Hide resolved

src/Turing.jl Show resolved Hide resolved

test/variational/advi.jl Outdated Show resolved Hide resolved

test/variational/vi.jl Show resolved Hide resolved

Red-Portal changed the base branch from main to breaking October 24, 2025 02:03

Red-Portal and others added 8 commits October 24, 2025 00:00

Merge branch 'breaking' of github.com:TuringLang/Turing.jl into bump_…

3e30e04

…advancedvi_0.5

update docs for vi

d870045

add history entry for AdvancedVI@0.5

2d928e0

remove export for removed symbol

5211b37

fix formatting

f0d615d

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix formatting

1b2351f

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

tidy tests advi

2be31b4

fix rename file advi.jl to vi.jl to reflect naming changes

e48ae42

Red-Portal requested a review from mhauru October 24, 2025 04:30

mhauru requested changes Oct 24, 2025

View reviewed changes

HISTORY.md Outdated Show resolved Hide resolved

HISTORY.md Outdated Show resolved Hide resolved

HISTORY.md Outdated Show resolved Hide resolved

HISTORY.md Outdated Show resolved Hide resolved

Red-Portal and others added 5 commits October 24, 2025 22:38

fix docs

44f7762

Co-authored-by: Markus Hauru <markus@mhauru.org>

fix HISTORY.md

fd0e928

Co-authored-by: Markus Hauru <markus@mhauru.org>

fix HISTORY.md

77276bd

Co-authored-by: Markus Hauru <markus@mhauru.org>

Merge branch 'main' of github.com:TuringLang/Turing.jl into bump_adva…

cb1620c

…ncedvi_0.5

update history

e70ddb4

Red-Portal and others added 7 commits November 21, 2025 10:01

Merge branch 'bump_advancedvi_0.5' of github.com:TuringLang/Turing.jl…

61e59a6

… into bump_advancedvi_0.5

add batch-and-match

4a039dd

fix format api table

f782f56

fix use fullrank Gaussian in tests since all algorithm support it

5251ee7

fix tweak step sizes, remove unused kwargs

b665f96

fix increase budgets for failing algorithms

15af544

run formatter

7711e42

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Red-Portal requested review from mhauru and penelopeysm November 21, 2025 18:14

rename main variational inference file to match module name

a319962

mhauru requested changes Nov 27, 2025

View reviewed changes

Red-Portal and others added 12 commits November 27, 2025 15:27

run formatter

6e76afa

Co-authored-by: Markus Hauru <markus@mhauru.org>

fix docstring

2e78774

run formatter

98d07f8

Co-authored-by: Markus Hauru <markus@mhauru.org>

update docstring

cc2cac9

Co-authored-by: Markus Hauru <markus@mhauru.org>

update docstring

2d88ddd

Co-authored-by: Markus Hauru <markus@mhauru.org>

fix missing docstring for unconstrained keyword argument

025202f

fix relax assert transformed dist to an exception and add test

59fe8dc

Update HISTORY.md

b1aa675

Co-authored-by: Markus Hauru <markus@mhauru.org>

run formatter

22895b0

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

733f0be

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

2530f05

Co-authored-by: Markus Hauru <markus@mhauru.org>

fix document the change in output of vi in HISTORY

cb5a798

Red-Portal requested a review from mhauru November 27, 2025 21:03

Red-Portal and others added 5 commits November 27, 2025 16:06

run formatter

8bd0ba7

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

ed25920

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

6be562d

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

df1b832

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

add missing namespace

d83148b

		If the chosen variational inference algorithm operates in an unconstrained space, then the provided initial variational approximation `q` must be a `Bijectors.TransformedDistribution` of an unconstrained distribution.
		For example, the initialization supplied by `q_meanfield_gaussian`,`q_fullrank_gaussian`, `q_locationscale`.

Update variational inference interface to match AdvancedVI@0.6 #2699

Are you sure you want to change the base?

Update variational inference interface to match AdvancedVI@0.6 #2699

Conversation

Red-Portal commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

codecov bot commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mhauru left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

penelopeysm commented Oct 24, 2025

Uh oh!

Red-Portal commented Oct 24, 2025

Uh oh!

penelopeysm commented Oct 24, 2025

Uh oh!

Red-Portal commented Nov 21, 2025

Uh oh!

mhauru left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhauru Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

Red-Portal Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mhauru Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

Red-Portal Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mhauru Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

Red-Portal Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Update variational inference interface to match `AdvancedVI@0.6` #2699

Update variational inference interface to match `AdvancedVI@0.6` #2699

Red-Portal commented Oct 22, 2025 •

edited

Loading

codecov bot commented Oct 22, 2025 •

edited

Loading

mhauru left a comment •

edited

Loading